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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a}. In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)^ Responsive to communication(s) filed on 14 September 2007 . 
2a)n This action is FINAL. 2b)K This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-5 and 7-10 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) ^ Claim (s) 1-5 and 7-10 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10)^ The drawing{s) filed on 02 October 2003 is/are: a)KI accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction Is required if the drawlng(s) is objected to. See 37 CFR 1.121(d). 
1 The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)13 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
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2. n Certified copies of the priority documents have been received in Application No. . 
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application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

Continued Examination Under 37 CFR LI 14 
A request for continued examination under 37 CFR 1.1 14, including the fee set forth in 37 CFR 
1.17(e), was filed in this application after final rejection. Since this application is eligible for 
continued examination under 37 CFR 1.1 14, and the fee set forth in 37 CFR 1.17(e) has been 
timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 
1.114. Applicant's submission filed on 9/14/2007 has been entered. 

Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office acfion: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 1-3, 5, and 7-10 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Olympus (Japanese Publication Number 06-070267, pub Mar 11, 1994) in view of Petrich (US 
Pub 2006/0280368 A 1). 

As per claim 1, Olympus teaches the claimed: 

1 . A three-dimensional image display method (in figure 2 and in the English translation of the 
abstract, where it states that the device produces image data to be displayed) comprising: 
detecting directions of incident light emitted from a light source at a plurality of detectors; (in 
figure 3y the photodetectors 11, and [0008] of the English Translation, where two or more 
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photodetectors analyze and determine the direction of the lighting of the external world by a 
processing circuit); 

displaying the three-dimensional image with the shadow (in figure 2 and in the English 
translation of paragraph [0006]), 

Olympus does not explicitly teach the remaining claim limitations. 
Olympus suggests the claimed: 

comparing the position of the light source and a virtual position of a display object in a three 
dimensional image displayed in real space to obtain a shadow for applying to the display object 
from a direction of the light source, the shadow being caused by the light source; ([0006] and 
[0008] of the English Translation which teaches of generating the image in figure 2b according 
to the detected light directions and shading their respective synthetic image according to the 
detected lighting directions so that the synthetic image generated is in agreement with the 
detected light directions); 

In this instance the direction of a light source and a virtual position of a display object are 
compared in Olympus. It would have been obvious to modify Olympus to include the claimed 
comparing the position of a light source with the display object in order to get a more accurate 
understanding of the shadow creation process when comparing the position of light source and 
the virtual position of a display object in real space. 

Petrich teaches the claimed: 
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calculating a position of the light source existing in real space based on the detected directions ( 
[0034], "FIGS. 8, P, 10, and 11 illustrate examples of how light qualities such as relative 
direction in relation to camera position, degree of diffusion, and degree of ambiance are 
ascertained from a digital image " and [0036], "Mathematical algorithms may also be used in 
place of a know table of values to calculate the position of light *'). 

It would have been obvious to one of ordinary skill in the art at the tinie of invention to combine 
Olympus with Petrich in order to obtain more accurate information relating to the light source 
such as a the position in addition to the direction. 

As per claim 2, Olympus teaches the claimed: 

2. The method according to claim 1, further comprising: 

detecting lightness of the light source at the detectors ([0008] of the English Translation where 
the lightness is detected at photodetectors 11). 

As per claim 3, the reasons and rationale for the rejection of claim 1 is incorporated herein. 
Olympus teaches the claimed: 

detecting directions of incident light emitted from a plurality of light sources at a plurality of 
detectors ([0008] of the English Translation where it states the photodetectors can be used to 
detect the direction of indoor lighting, thus the system in a room indoors with two lights would 
detect the indoor lighting direction of a plurality of light sources). 
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As per claim 5, this claim is similar in scope to claim 1, and thus is rejected under the same 
rationale. 

As per claim 7, Olympus teaches the claimed: 

7. The device according to claim 5, further comprising: a display surface configured to display 
the three-dimensional image, wherein: the direction detectors are disposed on at least one of the 
display surface and a surface adjacent to the display surface, (in figure 1 where the detectors 11 
are both on a display surface and adjacent to the display surface where these detectors are part 
of the head mount display where the display surface is the portion that covers the eyes of a user 
when wearing the device). 

As per claim 8, Olympus teaches the claimed: 

8. The device according to claim 5, further comprising: a display surface configured to display 
the three-dimensional image, wherein: the direction detectors are disposed to be adjacent to the 
display surface (in figure 1 where the detectors 11 are adjacent to the display surface which is 
the part of the head mounted display where the display surface is the portion that covers the eyes 
of a user when wearing the device). 

As per claim 9, Olympus teaches the claimed: 

9. The device according to claim 5, wherein the direction detectors are disposed at a position 
where the direction detectors detect the light emitted from the light source located in the same 
direction as at least one of a display direction of the three dimensional image and a direction in 
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which the three-dimensional image is observed (in figure 1 where it shows detectors 11 can be 
disposed at a direction the same as a display direction of the head-mounted display in relation to 
the user). 

As per claim 10, Olympus does not explicitly teach the claimed limitations. 
Petrich teaches the claimed: 

10. The device according to claim 5, wherein: each of the direction detectors include three- 
primary colors detection unit that adds colors to the shade. ([0030], '' other methods of detectins 
said target devices within a digital image include the use of: HSL, HSV, LAB, RGB and other 
digital image color spaces " where RGB is a red, green, and blue color sequence). 
It would have been obvious to one of ordinary skill in the art to use the claimed feature with 
Olympus in order to more accurate capture color data for analysis which can be used to produce 
better synthetic images. 

3. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Olympus 
(Japanese Publication Number 06-070267, pub Mar 11, 1994) in view of Petrich (US Pub 
2006/0280368 Al) in view of Drettakis et al. (NPL Document "Interactive Common 
Illumination for Computer Augmented Reality", herein referred to as "Drettakis"). 

As per claim 4, Olympus does not explicitly teach the claimed limitations. 
Drettakis teaches the claimed: 
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obtaining a position of a single virtual fight source, which represents the plurality of light 
sources; (at the top of the page where section 6.3 starts, "In addition, special attention must be 
taken in the re-scaling of the image before display since the addition of a source can add an 
order (or orders) of magnitude to the radiosity values of the scene '\ Here, the single virtual 
light source can be represented as the accumulation of radiosity. The reference refers to 
adding" light onto the total radiosity for each light source. When every light source is 
considered, one can represent this radiosity as a single virtual light source through the addition 
of all individual light sources) 

comparing the position of the virtual light source and the virtual position of the display object in 
the three-dimensional image to obtain a virtual shadow for applying to the display object from a 
direction of the single virtual light source, the virtual shadow being caused by the single virtual 
light source (in figures 3 and 4. Figure 4 shows the result of the comparison between the virtual 
light source (as calculated as the accumulated radiosity in figure 3) and the virtual position of 
the display object (the floating box above the desk). The comparison of the positions and 
relative positional relationship result in the rendering of a soft and varying shadow on the desk 
below the floating object. The varying nature of this shadow can be due to the plurality of light 
directional components provide by the virtual light source). 

It would have been obvious to one of ordinary skill in the art at the time of invention to combine 
Olympus, Petrich, and Drettakis. Drettakis teaches one advantage of the combination (in the 
abstract, "Our new framework will hopefiiUy lead to CAR systems with interactive common 
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illumination without restrictions on the movement of real or synthetic objects, lights and 
cameras" where the combining of light sources in Drettakis helps accomplish this stated goal by 
making it easier to do interactive illumination without restrictions). 

Response to Arguments 

Applicant's arguments with respect to the claims have been considered but are moot in view of 
the new ground(s) of rejection. 



Conclusion 

Any inquiry concerning this communication or earlier commimications from the 
examiner should be directed to Daniel F. Hajnik whose telephone number is (571) 272-7642. 
The examiner can normally be reached on Mon-Fri (8:30A-5:00P). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka J. Chauhan can be reached on (571) 272-7782. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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